\chapter{Transactions}
\label{chap:transactions}

HyperDex Warp is a superset of HyperDex that provides transactions.  For this
chapter, you'll need to install the \code{hyperdex-warp} demo package available
by contacting the HyperDex support team.

By the end of this chapter you'll be familiar with HyperDex Warp's transactions.

\section{Setup}
\label{sec:transactions:setup}

As in the previous chapter, the first step is to deploy the cluster and connect
a client.   First we launch and initialize the coordinator:

\begin{consolecode}
hyperdex coordinator -f -l 127.0.0.1 -p 1982
\end{consolecode}

Next, let's launch a few daemon processes to store data.  Execute the following
commands (note that each instance binds to a different port and has a different
\code{/path/to/data}):

\begin{consolecode}
hyperdex daemon -f --listen=127.0.0.1 --listen-port=2012 \
                   --coordinator=127.0.0.1 --coordinator-port=1982 --data=/path/to/data1
hyperdex daemon -f --listen=127.0.0.1 --listen-port=2013 \
                   --coordinator=127.0.0.1 --coordinator-port=1982 --data=/path/to/data2
hyperdex daemon -f --listen=127.0.0.1 --listen-port=2014 \
                   --coordinator=127.0.0.1 --coordinator-port=1982 --data=/path/to/data3
\end{consolecode}

This brings up three different daemons ready to serve in the HyperDex cluster.
Finally, we create a space which makes use of all three systems in the cluster.
In this example, let's create a space that may be suitable for storing virtual
currency in a banking application:

\begin{pyconcode}
>>> import hyperdex.client
>>> c = hyperdex.client.Client('127.0.0.1', 1982)
>>> c.add_space('''
... space accounts
... key id
... attribute
...    int balance
... ''')
\end{pyconcode}

Let's create some accounts with some starting balances:

\begin{pyconcode}
>>> c.put('accounts', 'joe', {'balance': 100})
True
>>> c.put('accounts', 'brian', {'balance': 25})
True
\end{pyconcode}

\section{Using Transactions}
\label{sec:transactions:using}

The prototypical example of where transactions come in handy is a standard bank
debit/credit scenario.  If Joe wanted to transfer currency to Brian, it would be
bad for Joe and Brian if the money were to leave Joe's account and not find its
way into Brian's.  It would be bad for the bank if money were to be created out
of thin air by a deposit in Brian's account without a corresponding withdrawal
from Joe's -- manipulating markets in such a fashion is reserved for those who
are too big to fail.

In the prototypical, "Hello World," example involving transactions, we transfer
\$10 from Joe to Brian:

\begin{pyconcode}
>>> x = c.begin_transaction()
>>> brian = x.get('accounts', 'brian')['balance']
>>> joe = x.get('accounts', 'joe')['balance']
>>> x.put('accounts', 'brian', {'balance': brian + 10})
True
>>> x.put('accounts', 'joe', {'balance': joe - 10})
True
>>> x.commit()
True
>>> print "Brian's balance =", c.get('accounts', 'brian')['balance']
Brian's balance = 35
>>> print "Joe's balance =", c.get('accounts', 'joe')['balance']
Joe's balance = 90
\end{pyconcode}

This transaction is relatively straight-forward.  There are three critical
properties that will distinguish this transaction.  One, either both of these
operations execute, or neither do.  It is impossible for half of a transaction
to be lost.

Two, there is no eventual consistency.  All transaction results are immediately
visible to all future events.  There is no need to reconcile or maintain
multiple versions.  There is no inconsistency.

Finally, HyperDex Warp is fault tolerant.  A user-configurable fault tolerance
threshold, \code{f} allows HyperDex Warp to remain available in the presence of
concurrent failures.

For a real bank application, we'd likely want to charge Joe an (excessive)
overdraft fee if he didn't have the money to cover the transfer.  With
transactions, implementing this case is trivial:

\begin{pyconcode}
>>> x = c.begin_transaction()
>>> brian = x.get('accounts', 'brian')['balance']
>>> joe = x.get('accounts', 'joe')['balance']
>>> x.put('accounts', 'brian', {'balance': brian + 10})
True
>>> if joe < 10:
...     # assess an overdraft fee on Joe
...     joe -= 35
...
>>> x.put('accounts', 'joe', {'balance': joe - 10})
True
>>> x.commit()
True
>>> print "Brian's balance =", c.get('accounts', 'brian')['balance']
Brian's balance = 45
>>> print "Joe's balance =", c.get('accounts', 'joe')['balance']
Joe's balance = 80
\end{pyconcode}

Here, we've successfully transferred money from Joe to Brian and cover the case
where the bank wishes to charge Joe a fee for not having enough money in the
first place.

The astute reader will notice that the example above is very different from
doing the following:

\begin{pyconcode}
>>> # an example of broken code
>>> c.atomic_add('accounts', 'brian', {'balance': 10})
True
>>> c.atomic_sub('accounts', 'joe', {'balance': 10})
True
>>> print "Brian's balance =", c.get('accounts', 'brian')['balance']
Brian's balance = 55
>>> print "Joe's balance =", c.get('accounts', 'joe')['balance']
Joe's balance = 70
\end{pyconcode}

This example is broken in multiple ways.  First, even though each individual
operation is atomic, the two operations are not indivisible, and will be
interspersed with all other operations on the accounts.  Second, the client is a
single point of failure for this transaction.  Imagine if the server executing
these operations failed between the two atomic operations.  No matter what the
order of operations, someone would lose money.

Transactions are especially useful when multiple competing processes are
executing transactions concurrently.  HyperDex Warp guarantees *one-copy
serializability*.  Any number of transactions may execute simultaneously.
The final state of the database will always be identical to executing the
committed transactions sequentially without concurrency.

Let's illustrate HyperDex's behavior with concurrent transactions.
In the same terminal, let's begin a new transaction to pay Joe his yearly
5\% interest payment:

\begin{pyconcode}
>>> x = c.begin_transaction()
>>> balance = x.get('accounts', 'joe')['balance']
>>> balance = int(balance * 1.05)
>>> x.put('accounts', 'joe', {'balance': balance})
True
\end{pyconcode}

At this point, transaction \code{x} has not committed.  Any modifications performed
within the context of a transaction are visible only within the transaction.  No
other clients or transactions may observe the state.  Verify this for yourself
with:

\begin{pyconcode}
>>> print "Joe's balance =", c.get('accounts', 'joe')['balance']
Joe's balance = 70
>>> print "Joe's balance =", x.get('accounts', 'joe')['balance']
Joe's balance = 73
\end{pyconcode}

Joe's balance is still the same as before.  Now, open another window and start a
second, concurrent transaction where Joe deposits cash at a branch location:

\begin{pyconcode}
>>> import hyperdex.client
>>> c2 = hyperdex.client.Client('127.0.0.1', 1982)
>>> y = c2.begin_transaction()
>>> balance = y.get('accounts', 'joe')['balance']
>>> balance += 386
>>> y.put('accounts', 'joe', {'balance': balance})
True
\end{pyconcode}

At this point, both \code{x} and \code{y} are ready to commit, but neither has
actually altered the account balance.  In fact, there is no way for \code{x} and
\code{y} to commit that doesn't violate consistency.  If \code{x} commits before
\code{y}, the ending balance will be \$466, shorting Joe of the \$4 in interest
he is due.  If, however, \code{y} commits before \code{x}, then the ending
balance of \$84 will cause Joe's cash deposit of \$386 to be lost into the
ether.  In this situation, one transaction must abort.  If \code{y} commits
first, then \code{x} will abort:

In the second terminal, commit \code{y}:

\begin{pyconcode}
>>> y.commit()
True
>>> print "Joe's balance =", c2.get('accounts', 'joe')['balance']
Joe's balance = 456
\end{pyconcode}

Trying to commit \code{x} in the first terminal will result in the transaction
being aborted by HyperDex Warp:

\begin{pyconcode}
>>> x.commit()
Traceback (most recent call last):
HyperClientException: HyperClient(HYPERCLIENT_ABORTED, Transaction was aborted)
\end{pyconcode}

As expected, transaction \code{x} will not alter Joe's balance:

\begin{pyconcode}
>>> print "Joe's balance =", c2.get('accounts', 'joe')['balance']
Joe's balance = 456
\end{pyconcode}

A transaction will only abort if it was executed simultaneously with another
transaction that operates on the same data.  The typical process for handling
aborted transactions is to repeatedly try the transaction until it succeeds:

\begin{pyconcode}
>>> committed = False
>>> while not committed:
...     try:
...         x = c.begin_transaction()
...         balance = x.get('accounts', 'joe')['balance']
...         balance = int(balance * 1.05)
...         x.put('accounts', 'joe', {'balance': balance})
...         committed = x.commit()
...     except hyperdex.client.HyperDexClientException as e:
...         if e.status != hyperdex.client.HYPERCLIENT_ABORTED:
...             raise e
...
True
\end{pyconcode}

\section{Rich API}
\label{sec:transactions:API}

HyperDex Warp Transactions expose the full set of atomic and asynchronous
operations from the HyperDex API.  All operations performed under a transaction
are fully transactional.  The following example shows asynchronous atomic math
operations combined with an asynchronous commit:

\begin{pyconcode}
>>> t = c.begin_transaction()
>>> d1 = t.async_atomic_add('accounts', 'brian', {'balance': 10})
>>> d2 = t.async_atomic_sub('accounts', 'joe', {'balance': 10})
>>> d1.wait()
True
>>> d2.wait()
True
>>> d3 = t.async_commit()
>>> d3.wait()
True
\end{pyconcode}

All transactions operate on the most recent copy of the data.  When a
transaction commits, the transaction is consistently applied.  HyperDex Warp
really means it when it says a transaction has committed.  There are no
conflicting operations to reconcile later (after the system commits), and there
is no eventual consistency.  All effects are immediate and permanent.

\section{Summary}
\label{sec:transactions:summary}

HyperDex Warp provides decentralized, distributed ACID transactions.  Committed
transactions are one-copy serializable, making it extremely easy to reason about
the concurrent operations.  The rich API coupled with transactional guarantees
provide a rare combination of ease-of-use, correctness, and performance that
other NoSQL systems cannot provide.
